The rapid increase in popularity of the World Wide Web (WWW or web) has led to a corresponding increase in the amount of traffic over the Internet. As a result, the web has become a primary bottleneck on network performance. When documents or information are requested by a user who is connected to a server via a slow network link, there can be noticeable latency at the user end. Further, transferring the document or information over the network leads to an increase in the level of traffic over the network. This reduces the bandwidth available for other requests. One way to reduce access latencies is to cache copies of popular documents or information closer to the user, from which the access latencies are more acceptable.
The cache can be implemented at various points on the network. For example, a large university or corporate network may have its own local cache, from which all subscribing users may fetch documents. In some cases, specialized servers called caching proxies, acting as agents on the behalf of the client are implemented in the network to locate a cached copy of a document. Typically, caching proxies serve as secondary or higher level caches, because they are concerned only with cache-misses from (primary) client caches. Client caches are typically part of the web browser, and may store either documents accessed during the current invocation (a nonpersistent cache such as is implemented by Mosaic), or documents accessed across invocations.
Generally speaking, a hierarchy of caches can be formed by the client and server(s). For example, in a corporate network, there can be one or more of a project proxy, a departmental proxy, a divisional proxy and a site proxy, etc. An Internet service provider can implement proxies at one or more of each neighborhood, each sub-region, and each region, etc. The client and/or proxies form a caching hierarchy. In a strict hierarchy, when a cache miss occurs, the (client or) proxy requests the missed object from the immediate higher level of the hierarchy through a caching proxy interface (identical to the HTTP interface as used in the CERN HTTP cache). More recently, in Harvest, "sibling" or "neighborhood" caches may be interrogated upon a cache-miss (see C. M. Brown, et. al., "Harvest: A Scalable, Customizable Discovery and Access System," in Technical Report CU-CS-732-94, Department of Computer Science, University of Colorado, 1994). In either case, the caching decision is made at each local proxy independent of objects cached in other proxies. In other words, caching decisions are made solely as a function of the local cache contents and/or object characteristics.
Thus, there is a need for an improved caching method and system in which the client and/or server(s) collaborate in making caching decisions and in the location of previously cached objects. The present invention addresses such a need.